Some known messaging systems provide persistent messaging, in which message state information and messages are saved to logs and message queue data files in persistent storage (such as disk storage). Persistently stored messages are able to survive most failures and restarts of the messaging system. In response to a failure other than a disk failure, the state information and messages can be recovered from the logged data and persistently-stored queues. The recoverability of persistent messages and state information is a significant factor in achieving assured once-only message delivery.
For example, a message queue manager may save a persistent message to disk storage before confirming to an application program that an operation has been successfully performed on the message. In a typical messaging network, a message-sending application program issues a ‘put_message’ instruction via an API call, to place an outgoing message in a queue. A local queue manager program manages this queue and communicates with other local application programs, or other queue managers within a distributed messaging network, to transfer the message to a target recipient. In a persistent messaging system, the local queue manager saves the message to persistent storage before confirming to the sender application that the ‘put_message’ operation completed successfully. A persistent message may be written to disk for every ‘put_message’ and ‘get_message’ that is performed to transfer the message to its destination (in some implementations, where the message is outside the current syncpoint-manager controlled ‘unit of work’), or the message may only be logged when a current unit of work is committed. With distributed units of work, it may be possible to avoid writing logs at a number of intermediate points during a message transfer, but a message that is identified as ‘persistent’ will be written to persistent storage at a pre-defined point.
In contrast, non-persistent messages are discarded when a queue manager experiences a failure and has to restart, as well as when the queue manager is stopped by an operator command. Although persistent messaging provides great advantages in terms of recoverability and assured message delivery, persistent messaging also has a performance cost. Writing to disk for every ‘put_message’ and ‘get_message’ operation will reduce message throughput and increase application response times, so there is a business justification for handling some messages non-persistently.
Because of the tradeoff between assured message delivery and performance, the WebSphere MQ family of products from IBM Corporation include support for setting persistence or non-persistence as optional attributes of a message and considerable work has been done to improve performance of persistent messaging (WebSphere and IBM are trademarks of International Business Machines Corporation). Performance has been enhanced by use of high performance disk storage, commit processing techniques that optimize performance in the absence of failures, and efficient recovery techniques to deal with failures. For example, U.S. Pat. No. 5,452,430, filed on 23 Mar. 1994 and assigned to IBM Corporation, describes an approach to handling persistent and non-persistent messages that aims to reduce delays during recovery from a system failure, by storing persistent and non-persistent data in separate sets of pages within a single message queue. C. Mohan and D. Dievendorff, “Recent Work on Distributed Commit Protocols, and Recoverable Messaging and Queuing” IEEE Data Engineering Bulletin, 17 (1), pages 22-28, 1994, describes developments in the areas of distributed commit protocols and recoverable messaging and queuing.
Nevertheless, many messaging systems still invest too much of their resources on ensuring message integrity, and typical messaging systems still have a relatively inflexible specification of persistence. One proposal is to give messaging system users an opportunity to specify a different persistence policy for different system states. In such a messaging system, users would benefit from an increased granularity of control over persistence but significant processing costs would remain and the persistence behaviour is fixed when the persistence policy is specified.